Delays in accessing high-quality care for newborns in East Africa: An analysis of survey data in Malawi, Mozambique, and Tanzania.

Background: Despite the existence of evidence-based interventions, substantial progress in reducing neonatal mortality is lagging, indicating that small and sick newborns (SSNs) are likely not receiving the care they require to survive and thrive. The 'three delays model' provides a framework for understanding the challenges in accessing care for SSNs. However, the extent to which each delay impacts access to care for SSNs is not well understood. To fill this evidence gap, we explored the impact of each of the three delays on access to care for SSNs in Malawi, Mozambique, and Tanzania. Methods: Secondary analyses of data from three different surveys served as the foundation of this study. To understand the impact of delays in the decision to seek care (delay 1) and the ability to reach an appropriate point of care (delay 2), we investigated time trends in place of birth disaggregated by facility type. We also explored care-seeking behaviours for newborns who died. To understand the impact of delays in accessing high-quality care after reaching a facility (delay 3), we measured facility readiness to manage care for SSNs. We used this measure to adjust institutional delivery coverage for SSN care readiness. Results: Coverage of institutional deliveries was substantially lower after adjusting for facility readiness to manage SSN care, with decreases of 30 percentage points (pp) in Malawi, 14 pp in Mozambique, and 24 pp in Tanzania. While trends suggest more SSNs are born in facilities, substantial gaps remain in facilities' capacities to provide lifesaving interventions. In addition, exploration of care-seeking pathways revealed that a substantial proportion of newborn deaths occurred outside of health facilities, indicating barriers in the decision to seek care or the ability to reach an appropriate source of care may also prevent SSNs from receiving these interventions. Conclusions: Investments are needed to overcome delays in accessing high-quality care for the most vulnerable newborns, those who are born small or sick. As more mothers and newborns access health services in low- and middle-income countries, ensuring that life-saving interventions for SSNs are available at the locations where newborns are born and seek care after birth is critical.

Validation of maternal recall of number of antenatal care visits attended in rural Southern Nepal: a longitudinal cohort study.

OBJECTIVES: This study aimed to examine the validity of maternal recall of total number of antenatal care (ANC) visits during pregnancy and factors associated with the accuracy of maternal recall. DESIGN: This was a longitudinal cohort study conducted from December 2018 through November 2020. SETTING: Five government health posts in the Sarlahi district of Southern Nepal. PARTICIPANTS: 402 pregnant women between ages 15 and 49 who presented for their first ANC visit at the study health posts. MAIN OUTCOMES: The observed number of ANC visits (gold standard) and the reported number of ANC visits at the postpartum interview (maternal recall). RESULTS: On average, women in the study who had a live birth attended 4.7 ANC visits. About 65% of them attended four or more ANC visits during pregnancy as recommended by the Nepal government, and 38.3% of maternal report matched the categorical ANC visits as observed by the gold standard. The individual validity was poor to moderate, with the highest area under the receiver operating characteristic curve (AUC) being 0.69 (95% CI: 0.65 to 0.74) in the 1-3 visits group. Population-level bias (as distinct from individual-level bias) was observed in the 1-3 visits and 4 visits groups, where 1-3 visits were under-reported (inflation factor (IF): 0.69) and 4 ANC visits were highly over-reported (IF: 2.12). The binary indicator ANC4+ (1-3 visits vs 4+ visits) showed better population-level validity (AUC: 0.69; IF: 1.17) compared with the categorical indicators (1-3 visits, 4 visits, 5-6 visits and more than 6 visits). Report accuracy was not associated with maternal characteristics but was related to ANC frequency. Women who attended more ANC visits were less likely to correctly report their total number of visits. CONCLUSION: Maternal report of number of ANC visits during pregnancy may not be a valid indicator for measuring ANC coverage. Improvements are needed to measure the frequency of ANC visits.

Association between antenatal care facility readiness and provision of care at the client level and facility level in five low- and middle-income countries.

BACKGROUND: Despite growing interest in monitoring improvements in quality of care, data on service quality in low-income and middle-income countries (LMICs) is limited. While health systems researchers have hypothesized the relationship between facility readiness and provision of care, there have been few attempts to quantify this relationship in LMICs. This study assesses the association between facility readiness and provision of care for antenatal care at the client level and facility level. METHODS: To assess the association between provision of care and various facility readiness indices for antenatal care, we used multilevel, multivariable random-effects linear regression models. We tested an inflection point on readiness scores by fitting linear spline models. To compare the coefficients between models, we used a bootstrapping approach and calculated the mean difference between all pairwise comparisons. Analyses were conducted at client and facility levels. RESULTS: Our results showed a small, but significant association between facility readiness and provision of care across countries and most index constructions. The association was most evident in the client-level analyses that had a larger sample size and were adjusted for factors at the facility, health worker, and individual levels. In addition, spline models at a facility readiness score of 50 better fit the data, indicating a plausible threshold effect. CONCLUSIONS: The results of this study suggest that facility readiness is not a proxy for provision of care, but that there is an important association between facility readiness and provision of care. Data on facility readiness is necessary for understanding the foundations of health systems particularly in countries with the lowest levels of service quality. However, a comprehensive view of quality of care should include both facility readiness and provision of care measures.

Exploring approaches to weighting estimates of facility readiness to provide health services used for estimating input-adjusted effective coverage: a case study using data from Tanzania.

The ideal approach for calculating effective coverage of health services using ecological linking requires accounting for variability in facility readiness to provide health services and patient volume by incorporating adjustments for facility type into estimates of facility readiness and weighting facility readiness estimates by service-specific caseload. The aim of this study is to compare the ideal caseload-weighted facility readiness approach to two alternative approaches: (1) facility-weighted readiness and (2) observation-weighted readiness to assess the suitability of each as a proxy for caseload-weighted facility readiness. We utilised the 2014-2015 Tanzania Service Provision Assessment along with routine health information system data to calculate facility readiness estimates using the three approaches. We then conducted equivalence testing, using the caseload-weighted estimates as the ideal approach and comparing with the facility-weighted estimates and observation-weighted estimates to test for equivalence. Comparing the facility-weighted readiness estimates to the caseload-weighted readiness estimates, we found that 58% of the estimates met the requirements for equivalence. In addition, the facility-weighted readiness estimates consistently underestimated, by a small percentage, facility readiness as compared to the caseload-weighted readiness estimates. Comparing the observation-weighted readiness estimates to the caseload-weighted readiness estimates, we found that 64% of the estimates met the requirements for equivalence. We found that, in this setting, both facility-weighted readiness and observation-weighted readiness may be reasonable proxies for caseload-weighted readiness. However, in a setting with more variability in facility readiness or larger differences in facility readiness between low caseload and high caseload facilities, the observation-weighted approach would be a better option than the facility-weighted approach. While the methods compared showed equivalence, our results suggest that selecting the best method for weighting readiness estimates will require assessing data availability alongside knowledge of the country context.

How does social desirability bias influence survey-based estimates of the use of antenatal care in rural Nepal? A validation study.

OBJECTIVES: Social desirability bias is often speculated to influence survey responses but seldom studied in healthcare. The objective was to explore whether social desirability scores (SDS) or the presence of interview observers is associated with inaccurate recall and overestimation of antenatal care (ANC) services. DESIGN: Longitudinal validation study comparing recalled receipt of ANC services and nutrition components of ANC against direct observations of care. An adapted short form Marlowe-Crowne questionnaire was used to generate an SDS, and the presence of interview observers was treated as a separate exposure. We assessed accuracy and overestimation of recalled receipt of ANC services against observed receipt using log-binomial regression, adjusting for age, education, first-pregnancy and socioeconomic status. SETTING: Rural Southern Nepal with recruitment from five government health posts. PARTICIPANTS: 401 pregnant women. RESULTS: Social desirability scores did not significantly predict accuracy or overestimation of most types of ANC care except counselling on nausea. Higher SDS was associated with more accurate recall (adjusted RR, aRR 1.08 (95% CI 1.03, 1.12)) and less overestimation (aRR 0.85 (0.80, 0.91)). The presence of mothers-in-law or husbands during interviews was associated with greater overestimation of the number of ANC visits received by more than three visits (aRR 2.07 (1.11, 3.84)) and (aRR 4.19 (2.17, 8.10)), respectively. Those interviewed with friends present tended to overestimate the receipt of counselling on nausea, avoiding alcohol and not smoking. CONCLUSION: The presence of observers can lead to overestimation of the receipt of ANC care and support the conduct of interviews in private settings despite challenges of doing so in village contexts. Findings that the SDS did not predict the accuracy of most types of ANC care might reflect a reality that such questions may not be sensitive from a social-norms perspective. Additional local adaptation of SDS is recommended.

Moderate Accuracy of Survey Responses about Infant and Young Child Feeding Counseling Reported by Mothers with Children Less than 1 Year of Age in India.

BACKGROUND: Counseling on infant and young child feeding (IYCF) to support optimal breastfeeding and complementary feeding practices is an essential intervention, and accurate coverage data is needed to identify gaps and monitor progress. However, coverage information captured during household surveys has not yet been validated. OBJECTIVES: We examined the validity of maternal reports of IYCF counseling received during community-based contacts and factors associated with reporting accuracy. METHODS: Direct observations of home visits conducted by community workers in 40 villages in Bihar, India served as the "gold standard" to maternal reports of IYCF counseling received during 2-wk follow-up surveys (n = 444 mothers with children less than 1 y of age, interviews matched to direct observations). Individual-level validity was assessed by calculating sensitivity, specificity, and AUC. Population-level bias was measured using the inflation factor (IF). Multivariable regression models were used to examine factors associated with response accuracy. RESULTS: Prevalence of IYCF counseling during home visits was very high (90.1%). Maternal report of any IYCF counseling received in the past 2 wk was moderate (AUC: 0.60; 95% CI: 0.52, 0.67), and population bias was low (IF = 0.90). However, the recall of specific counseling messages varied. Maternal report of any breastfeeding, exclusive breastfeeding, and dietary diversity messages had moderate validity (AUC > 0.60), but other child feeding messages had low individual validity. Child age, maternal age, maternal education, mental stress, and social desirability were associated with reporting accuracy of multiple indicators. CONCLUSIONS: Validity of IYCF counseling coverage was moderate for several key indicators. IYCF counseling is an information-based intervention that may be received from various sources, and it may be challenging to achieve higher reporting accuracy over a longer recall period. We consider the modest validity results as positive and suggest that these coverage indicators may be useful for measuring coverage and tracking progress over time.

The effect of sampling health facilities on estimates of effective coverage: a simulation study.

BACKGROUND: Most existing facility assessments collect data on a sample of health facilities. Sampling of health facilities may introduce bias into estimates of effective coverage generated by ecologically linking individuals to health providers based on geographic proximity or administrative catchment. METHODS: We assessed the bias introduced to effective coverage estimates produced through two ecological linking approaches (administrative unit and Euclidean distance) applied to a sample of health facilities. Our analysis linked MICS household survey data on care-seeking for child illness and childbirth care with data on service quality collected from a census of health facilities in the Savanes region of Cote d'Ivoire. To assess the bias introduced by sampling, we drew 20 random samples of three different sample sizes from our census of health facilities. We calculated effective coverage of sick child and childbirth care using both ecological linking methods applied to each sampled facility data set. We compared the sampled effective coverage estimates to ecologically linked census-based estimates and estimates based on true source of care. We performed sensitivity analyses with simulated preferential care-seeking from higher-quality providers and randomly generated provider quality scores. RESULTS: Sampling of health facilities did not significantly bias effective coverage compared to either the ecologically linked estimates derived from a census of facilities or true effective coverage estimates using the original data or simulated random quality sensitivity analysis. However, a few estimates based on sampling in a setting where individuals preferentially sought care from higher-quality providers fell outside of the estimate bounds of true effective coverage. Those cases predominantly occurred using smaller sample sizes and the Euclidean distance linking method. None of the sample-based estimates fell outside the bounds of the ecologically linked census-derived estimates. CONCLUSIONS: Our analyses suggest that current health facility sampling approaches do not significantly bias estimates of effective coverage produced through ecological linking. Choice of ecological linking methods is a greater source of bias from true effective coverage estimates, although facility sampling can exacerbate this bias in certain scenarios. Careful selection of ecological linking methods is essential to minimize the potential effect of both ecological linking and sampling error.

The RADAR coverage tool: developing a toolkit for rigorous household surveys for reproductive, maternal, newborn, and child health & nutrition indicators.

Population-based intervention coverage data are used to inform the design of projects, programs, and policies and to evaluate their impact. In low- and middle-income countries (LMICs), household surveys are the primary source of coverage data. Many coverage surveys are implemented by organizations with limited experience or resources in population-based data collection. We developed a streamlined survey and set of supporting materials to facilitate rigorous survey design and implementation. The RADAR coverage survey tool aimed to 1) rigorously measure priority reproductive, maternal, newborn, child health & nutrition coverage indicators, and allow for equity and gender analyses; 2) use standard, valid questions, to the extent possible; 3) be as light as possible; 4) be flexible to address users' needs; and 5) be compatible with the Lives Saved Tool for analysis of program impact. Early interactions with stakeholders also highlighted survey planning, implementation, and analysis as challenging areas. We therefore developed a suite of resources to support implementers in these areas. The toolkit was piloted by implementers in Tanzania and in Burkina Faso. Although the toolkit was successfully implemented in these settings and facilitated survey planning and implementation, we found that implementers must still have access to sufficient resources, time, and technical expertise in order to use the tool appropriately. This potentially limits the use of the tool to situations where high-quality surveys or evaluations have been prioritized and adequately resourced.

Cognitive testing of questions about antenatal care and nutrition interventions in southern Nepal.

Large scale surveys such as the Demographic and Health Surveys (DHS) are used to measure the coverage and quality of antenatal care (ANC)-related services. Studies have increasingly validated questions from these surveys, though few have explored respondent comprehension or associated thought processes. This study aimed to use cognitive testing and validation approaches to understand how survey respondents understand questions related to ANC-related nutrition services. The study was nested within a larger validation study in southern Nepal. Pregnant women's receipt of ANC related services was directly observed at five health posts followed by a recall interview at 6 months postpartum. A week later, a survey module was re-administered to 30 women containing 15 questions about receipt of ANC care and specifically nutrition-related services. Detailed probing was used to identify cognitive challenges related to comprehension, retrieval, judgement, and response. Respondents accurately recalled the four specific ANC visits recommended by the government of Nepal but those with more visits struggled to estimate the total number of ANC visits they had made. A number of terms including "antenatal care, "nutrition" and "breastfeeding" were challenging for many respondents to understand. Visits to private providers including for ultrasounds were inconsistently included in ANC visit counts suggesting that question wording could better specify the type of care. Many respondents over-estimated the number of iron folic acid (IFA) supplements taken during pregnancy, and recall was challenging. Calculations were based on estimating the number of months between first ANC visit to delivery, and only sometimes factored in missed tablets. Opportunities exist to improve questions to facilitate better comprehension by respondents through a combination of using local terms and explanations, reordering some questions, and adapting questions to better match respondents' approaches to estimating numeric responses.

Validation of MINORMIX Approach for Estimation of Low Birthweight Prevalence Using a Rural Nepal Dataset.

BACKGROUND: The Global Nutrition Target of reducing low birthweight (LBW) by ≥30% between 2012 and 2025 has led to renewed interest in producing accurate, population-based, national LBW estimates. Low- and middle-income countries rely on household surveys for birthweight data. These data are frequently incomplete and exhibit strong "heaping." Standard survey adjustment methods produce estimates with residual bias. The global database used to report against the LBW Global Nutrition Target adjusts survey data using a new MINORMIX (multiple imputation followed by normal mixture) approach: 1) multiple imputation to address missing birthweights, followed by 2) use of a 2-component normal mixture model to account for heaping of birthweights. OBJECTIVES: To evaluate the performance of the MINORMIX birthweight adjustment approach and alternative methods against gold-standard measured birthweights in rural Nepal. METHODS: As part of a community-randomized trial in rural Nepal, we measured "gold-standard" birthweights at birth and returned 1-24 mo later to collect maternally reported birthweights using standard survey methods. We compared estimates of LBW from maternally reported data derived using: 1) the new MINORMAX approach; 2) the previously used Blanc-Wardlaw adjustment; or 3) no adjustment for missingness or heaping against our gold standard. We also assessed the independent contribution of multiple imputation and curve fitting to LBW adjustment. RESULTS: Our gold standard found 27.7% of newborns were LBW. The unadjusted LBW estimate based on maternal report with simulated missing birthweights was 14.5% (95% CI: 11.6, 18.0%). Application of the Blanc-Wardlaw adjustment increased the LBW estimate to 20.6%. The MINORMIX approach produced an estimate of 26.4% (95% CI: 23.5, 29.3%) LBW, closest to and with bounds encompassing the measured point estimate. CONCLUSIONS: In a rural Nepal validation dataset, the MINORMIX method generated a more accurate LBW estimate than the previously applied adjustment method. This supports the use of the MINORMIX method to produce estimates for tracking the LBW Global Nutrition Target.

Advancing nutrition measurement: Developing quantitative measures of nutrition service quality for pregnant women and children in low- and middle-income country health systems.

The health sector plays an important role in the delivery of high-quality nutrition interventions to women and children in low- and middle-income countries (LMICs). However, there are no standardized approaches to defining and measuring nutrition service quality in these contexts. This study aims to systematically develop quality of care indices for direct health systems nutrition interventions using a five-step process: (1) identify recommended interventions for inclusion in indices, (2) extract service readiness, provision of care, and experience of care items from intervention-specific clinical guidelines, (3) map items to data available in global health facility surveys, (4) conduct an expert survey to prioritize interventions and items, and (5) use findings from previous steps to propose quality of care metrics. Thirty-two recommended interventions were identified, for which the guidelines review yielded 763 unique items that were reviewed by experts. The proposed nutrition quality of care indices for pregnant women reflects eight interventions and the indices for children under 5 reflects six interventions. The indices provide a standardized measure for nutrition intervention quality and can be operationalized using existing health facility assessment data, facilitating their use by LMIC decision makers for planning and resource allocation.

Using cognitive interviewing to bridge the intent-interpretation gap for nutrition coverage survey questions in India.

Designing survey questions that clearly and precisely communicate the question's intent and elicit responses based on the intended interpretation is critical but often undervalued. We used cognitive interviewing to qualitatively assess respondents' interpretation of and responses to questions pertaining to maternal and child nutrition intervention coverage. We conducted interviews to cognitively test 25 survey questions with mothers (N = 21) with children less than 1 year in Madhya Pradesh, India. Each question was followed by probes to capture information on four cognitive stages-comprehension, retrieval, judgement, and response. Data were analysed for common and unique patterns across the survey questions. We identified four types of cognitive challenges: (1) retention of multiple concepts in long questions: difficulty in comprehending and retaining questions with three or more key concepts; (2) temporal confusion: difficulty in conceptualizing recall periods such as "in the last 6 months" as compared to life stages such as pregnancy; (3) interpretation of concepts: mismatch of information being asked, meaning of certain terms and intervention scope; and (4) understanding of technical terms: difficulty in understanding commonly used technical words such as "breastfeeding" and "antenatal care" and requiring use of simple alternative language. Findings from this study will be useful for stakeholders involved in survey design and implementation, especially those conducting large-scale household surveys to measure coverage of essential nutrition interventions.

Methodological considerations for linking household and healthcare provider data for estimating effective coverage: a systematic review.

OBJECTIVE: To assess existing knowledge related to methodological considerations for linking population-based surveys and health facility data to generate effective coverage estimates. Effective coverage estimates the proportion of individuals in need of an intervention who receive it with sufficient quality to achieve health benefit. DESIGN: Systematic review of available literature. DATA SOURCES: Medline, Carolina Population Health Center and Demographic and Health Survey publications and handsearch of related or referenced works of all articles included in full text review. The search included publications from 1 January 2000 to 29 March 2021. ELIGIBILITY CRITERIA: Publications explicitly evaluating (1) the suitability of data, (2) the implications of the design of existing data sources and (3) the impact of choice of method for combining datasets to obtain linked coverage estimates. RESULTS: Of 3805 papers reviewed, 70 publications addressed relevant issues. Limited data suggest household surveys can be used to identify sources of care, but their validity in estimating intervention need was variable. Methods for collecting provider data and constructing quality indices were diverse and presented limitations. There was little empirical data supporting an association between structural, process and outcome quality. Few studies addressed the influence of the design of common data sources on linking analyses, including imprecise household geographical information system data, provider sampling design and estimate stability. The most consistent evidence suggested under certain conditions, combining data based on geographical proximity or administrative catchment (ecological linking) produced similar estimates to linking based on the specific provider utilised (exact match linking). CONCLUSIONS: Linking household and healthcare provider data can leverage existing data sources to generate more informative estimates of intervention coverage and care. However, existing evidence on methods for linking data for effective coverage estimation are variable and numerous methodological questions remain. There is need for additional research to develop evidence-based, standardised best practices for these analyses.

Impact of imprecise household location on effective coverage estimates generated through linking household and health provider data by geographic proximity: a simulation study.

BACKGROUND: Geographic proximity is often used to link household and health provider data to estimate effective coverage of health interventions. Existing household surveys often provide displaced data on the central point within household clusters rather than household location. This may introduce error into analyses based on the distance between households and providers. METHODS: We assessed the effect of imprecise household location on quality-adjusted effective coverage of child curative services estimated by linking sick children to providers based on geographic proximity. We used data on care-seeking for child illness and health provider quality in Southern Province, Zambia. The dataset included the location of respondent households, a census of providers, and data on the exact outlets utilized by sick children included in the study. We displaced the central point of each household cluster point five times. We calculated quality-adjusted coverage by assigning each sick child to a provider's care based on three measures of geographic proximity (Euclidean distance, travel time, and geographic radius) from the household location, cluster point, and displaced cluster locations. We compared the estimates of quality-adjusted coverage to each other and estimates using each sick child's true source of care. We performed sensitivity analyses with simulated preferential care-seeking from higher-quality providers and randomly generated provider quality scores. RESULTS: Fewer children were linked to their true source of care using cluster locations than household locations. Effective coverage estimates produced using undisplaced or displaced cluster points did not vary significantly from estimates produced using household location data or each sick child's true source of care. However, the sensitivity analyses simulating greater variability in provider quality showed bias in effective coverage estimates produced with the geographic radius and travel time method using imprecise location data in some scenarios. CONCLUSIONS: Use of undisplaced or displaced cluster location reduced the proportion of children that linked to their true source of care. In settings with minimal variability in quality within provider categories, the impact on effective coverage estimates is limited. However, use of imprecise household location and choice of geographic linking method can bias estimates in areas with high variability in provider quality or preferential care-seeking.

A call for standardised age-disaggregated health data.

The 2030 Sustainable Development Goals agenda calls for health data to be disaggregated by age. However, age groupings used to record and report health data vary greatly, hindering the harmonisation, comparability, and usefulness of these data, within and across countries. This variability has become especially evident during the COVID-19 pandemic, when there was an urgent need for rapid cross-country analyses of epidemiological patterns by age to direct public health action, but such analyses were limited by the lack of standard age categories. In this Personal View, we propose a recommended set of age groupings to address this issue. These groupings are informed by age-specific patterns of morbidity, mortality, and health risks, and by opportunities for prevention and disease intervention. We recommend age groupings of 5 years for all health data, except for those younger than 5 years, during which time there are rapid biological and physiological changes that justify a finer disaggregation. Although the focus of this Personal View is on the standardisation of the analysis and display of age groups, we also outline the challenges faced in collecting data on exact age, especially for health facilities and surveillance data. The proposed age disaggregation should facilitate targeted, age-specific policies and actions for health care and disease management.

Reliability of maternal recall of delivery and immediate newborn care indicators in Sarlahi, Nepal.

BACKGROUND: The intrapartum period is a time of high mortality risk for newborns and mothers. Numerous interventions exist to minimize risk during this period. Data on intervention coverage are needed for health system improvement. Maternal report of intrapartum interventions through surveys is the primary source of coverage data, but they may be invalid or unreliable. METHODS: We assessed the reliability of maternal report of delivery and immediate newborn care for a sample of home and health facility births in Sarlahi, Nepal. Mothers were visited as soon as possible following delivery (< 72 h) and asked to report circumstances of labor and delivery. A subset was revisited 1-24 months after delivery and asked to recall interventions received using standard household survey questions. We assessed the reliability of each indicator by comparing what mothers reported immediately after delivery against what they reported at the follow-up survey. We assessed potential variation in reliability of maternal report by characteristics of the mother, birth event, or intervention prevalence. RESULTS: One thousand five hundred two mother/child pairs were included in the reliability study, with approximately half of births occurring at home. A higher proportion of women who delivered in facilities reported "don't know" when asked to recall specific interventions both initially and at follow-up. Most indicators had high observed percent agreement, but kappa values were below 0.4, indicating agreement was primarily due to chance. Only "received any injection during delivery" demonstrated high reliability among all births (kappa: 0.737). The reliability of maternal report was typically lower among women who delivered at a facility. There was no difference in reliability based on time since birth of the follow-up interview. We observed over-reporting of interventions at follow-up that were more common in the population and under-reporting of less common interventions. CONCLUSIONS: This study reinforces previous findings that mothers are unable to report reliably on many interventions within the peripartum period. Household surveys which rely on maternal report, therefore, may not be an appropriate method for collecting data on coverage of many interventions during the peripartum period. This is particularly true among facility births, where many interventions may occur without the mother's full knowledge.

Assessment of facility and health worker readiness to provide quality antenatal, intrapartum and postpartum care in rural Southern Nepal.

BACKGROUND: Increased coverage of antenatal care and facility births might not improve maternal and newborn health outcomes if quality of care is sub-optimal. Our study aimed to assess the facility readiness and health worker knowledge required to provide quality maternal and newborn care. METHODS: Using an audit tool and interviews, respectively, facility readiness and health providers' knowledge of maternal and immediate newborn care were assessed at all 23 birthing centers (BCs) and the District hospital in the rural southern Nepal district of Sarlahi. Facility readiness to perform specific functions was assessed through descriptive analysis and comparisons by facility type (health post (HP), primary health care center (PHCC), private and District hospital). Knowledge was compared by facility type and by additional skilled birth attendant (SBA) training. RESULTS: Infection prevention items were lacking in more than one quarter of facilities, and widespread shortages of iron/folic acid tablets, injectable ampicillin/gentamicin, and magnesium sulfate were a major barrier to facility readiness. While parenteral oxytocin was commonly provided, only the District hospital was prepared to perform all seven basic emergency obstetric and newborn care signal functions. The required number of medical doctors, nurses and midwives were present in only 1 of 5 PHCCs. Private sector SBAs had significantly lower knowledge of active management of third stage of labor and correct diagnosis of severe pre-eclampsia. While half of the health workers had received the mandated additional two-month SBA training, comparison with the non-trained group showed no significant difference in knowledge indicators. CONCLUSIONS: Facility readiness to provide quality maternal and newborn care is low in this rural area of Nepal. Addressing the gaps by facility type through regular monitoring, improving staffing and supply chains, supervision and refresher trainings is important to improve quality.

Validity of self-reported receipt of iron supplements during pregnancy: implications for coverage measurement.

BACKGROUND: Iron-deficiency anemia during pregnancy is an underlying cause of maternal deaths, and reducing risk through routine iron supplementation is a key component of antenatal care (ANC) programs in most low- and middle income countries. Supplementation coverage during pregnancy is estimated from maternal self-reports in population-based household surveys, yet recall bias and social desirability bias lead to errors of unknown magnitude. METHODS: We linked data from household and health facility surveys from 16 countries to estimate input-adjusted coverage of iron supplementation during pregnancy. We assessed the validity of reported receipt of iron supplements in client exit interviews using direct observation as the gold standard across 9 countries with a recent Service Provision Assessment (SPA). Using a sample of 227 women who participated in the Nepal Oil Massage Study (NOMS), we also assessed the validity of self-reported receipt of iron folic acid (IFA) supplements. We used Poisson regression models to explore the association between client and health facility characteristics and agreement of self-reported receipt of iron supplements compared to direct observation. RESULTS: Across the 16 countries, iron supplements were in supply at most of the 9215 sampled health facilities offering ANC services (91%). We estimated that between 48 and 93% of women attended at least one ANC visit at a health facility with iron supplements available. The specificity of recall of receipt of iron supplementation immediately following a visit was 79.3% and the sensitivity was 88.7% for the entire sample. Individual-level accuracy was high (Area under the curve > 0.7) and population bias low (0.75 < inflation factor < 1.25) across all countries. By contrast, in the NOMS sub-study, the accuracy of self-reported receipt of IFA supplements after 1-2 years was poor (sensitivity 86.1%, specificity 34.3%). Adjusted regression analyses indicated that older age and higher level of education were associated with poorer agreement between self-reports and direct observation. CONCLUSIONS: These findings suggest the need for caution when using self-reported measures with an extended recall period. Further validation studies using conditions similar to widely used population-based household surveys are warranted.

Linking household survey and health facility data for effective coverage measures: a comparison of ecological and individual linking methods using the Multiple Indicator Cluster Survey in Côte d'Ivoire.

BACKGROUND: Population-based measures of intervention coverage are used in low- and middle-income countries for program planning, prioritization, and evaluation. There is increased interest in effective coverage, which integrates information about service quality or health outcomes. Approaches proposed for quality-adjusted effective coverage include linking data on need and service contact from population-based surveys with data on service quality from health facility surveys. However, there is limited evidence about the validity of different linking methods for effective coverage estimation. METHODS: We collaborated with the 2016 Côte d'Ivoire Multiple Indicator Cluster Survey (MICS) to link data from a health provider assessment to care-seeking data collected by the MICS in the Savanes region of Côte d'Ivoire. The provider assessment was conducted in a census of public and non-public health facilities and pharmacies in Savanes in May-June 2016. We also included community health workers managing sick children who served the clusters sampled for the MICS. The provider assessment collected information on structural and process quality for antenatal care, delivery and immediate newborn care, postnatal care, and sick child care. We linked the MICS and provider data using exact-match and ecological linking methods, including aggregate linking and geolinking methods. We compared the results obtained from exact-match and ecological methods. RESULTS: We linked 731 of 786 care-seeking episodes (93%) from the MICS to a structural quality score for the provider named by the respondent. Effective coverage estimates computed using exact-match methods were 13%-63% lower than the care-seeking estimates from the MICS. Absolute differences between exact match and ecological linking methods were ±7 percentage points for all ecological methods. Incorporating adjustments for provider category and weighting by service-specific utilization into the ecological methods generally resulted in better agreement between ecological and exact match estimates. CONCLUSIONS: Ecological linking may be a feasible and valid approach for estimating quality-adjusted effective coverage when a census of providers is used. Adjusting for provider type and caseload may improve agreement with exact match results. There remain methodological questions to be addressed to develop guidance on using linking methods for estimating quality-adjusted effective coverage, including the effect of facility sampling and time displacement.

Validation studies for population-based intervention coverage indicators: design, analysis, and interpretation.

BACKGROUND: Population-based intervention coverage indicators are widely used to track country and program progress in improving health and to evaluate health programs. Indicator validation studies that compare survey responses to a "gold standard" measure are useful to understand whether the indicator provides accurate information. The Improving Coverage Measurement (ICM) Core Group has developed and implemented a standard approach to validating coverage indicators measured in household surveys, described in this paper. METHODS: The general design of these studies includes measurement of true health status and intervention receipt (gold standard), followed by interviews with the individuals observed, and a comparison of the observations (gold standard) to the responses to survey questions. The gold standard should use a data source external to the respondent to document need for and receipt of an intervention. Most frequently, this is accomplished through direct observation of clinical care, and/or use of a study-trained clinician to obtain a gold standard diagnosis. Follow-up interviews with respondents should employ standard survey questions, where they exist, as well as alternative or additional questions that can be compared against the standard household survey questions. RESULTS: Indicator validation studies should report on participation at every stage, and provide data on reasons for non-participation. Metrics of individual validity (sensitivity, specificity, area under the receiver operating characteristic curve) and population-level validity (inflation factor) should be reported, as well as the percent of survey responses that are "don't know" or missing. Associations between interviewer and participant characteristics and measures of validity should be assessed and reported. CONCLUSIONS: These methods allow respondent-reported coverage measures to be validated against more objective measures of need for and receipt of an intervention, and should be considered together with cognitive interviewing, discriminative validity, or reliability testing to inform decisions about which indicators to include in household surveys. Public health researchers should assess the evidence for validity of existing and proposed household survey coverage indicators and consider validation studies to fill evidence gaps.